A Modified Orthant-Wise Limited Memory Quasi-Newton Method

where U = V k−mV k−m+1 · · ·V k−1. For the L-BFGS, we need not explicitly store the approximated inverse Hessian matrix. Instead, we only require matrix-vector multiplications at each iteration, which can be implemented by a twoloop recursion with a time complexity of O(mn) (Jorge & Stephen, 1999). Thus, we only store 2m vectors of length n: sk−1, sk−2, · · · , sk−m and yk−1,yk−2, · · · ,yk−m with a storage complexity of O(mn), which is very useful when n is large. In practice, L-BFGS updates Hk−m as μI , where μ = (s)y/∥y∥.